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DESCRIPTION 

IMAGE PROCESSING SYSTEM AND IMAGE PROCESSING METHOD 



Technical Field 

The present invention relates to an image processing 
method for performing image processing to an endoscopic 
image in order to easily evaluate the amount of feature. 

Background Art 

Endoscope systems are widely diffused. According to 
the endoscope system, a long insertion portion is inserted 
into a body cavity, an organ in the body cavity is observed 
on a monitor screen using a solid-state image pick-up device 
as image pick-up means, and the organ can be examined or 
diagnosed. Further, ultrasonic endoscope systems are also 
widespread. According to the ultrasonic endoscope system, 
an organ in the body cavity is irradiated with ultrasonic 
waves, and the situation of the organ is observed on a 
monitor screen using the reflectance or transmittance of the 
organ with the ultrasonic waves. Thus, the organ can be 
examined or diagnosed. 

The final diagnoses using the above-mentioned endoscope 
systems mostly depend on the subjectivities of doctors. It 
is desired that an endoscopic diagnosis support system 
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directly leading to an objective diagnosis using numeric 
data be realized. 

The endoscopic diagnosis support system supports an 
objective diagnosis with numeric data as follows. Various 
amounts of feature are calculated from regions of interest 
(ROD of an image of a diagnosis target. The image is 
categorized by threshold processing the amounts of feature 
or processing the amounts of feature using a statistical or 
hon- statistical discrimination circuit, thus determining a 
finding or a lesion. The finding or lesion based on the 
image is provided to a doctor. 

The amounts of feature indicate numeric values 
reflecting various findings of an endoscopic image and are 
obtained by an image processing method. When a finding 
related to a color tone, for example, a case where the 
surface of mucous membrane is red by rubefaction is 
converted into the amount of feature, R, G, B data 
constituting an endoscopic image are used, R/ (R+G+B) is 
obtained every pixel, and the average can be used as the 
amount of feature (the amount of feature is generally called 
chromatid ty) . In the recent endoscopic field, as the 
amount of feature of a color tone reflecting the bloodstream 
through a gastric mucosa, hemoglobin index obtained by 
32log2(R/G) is generally used. 

In addition, findings related to the surface structure 
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of a mucosa in an endoscopic image, for example, the 
deflation or meandering of a blood vessel observed in a 
visible blood vessel image, a variation in size of gastric 
areas, irregularity of gastric area, and the width of a 
groove between gastric areas are important elements for 
diagnoses of various diseases. These elements are processed 
by an image processing method, thus obtaining numeric values 
as the amounts of feature- Japanese Patent No. 2918162 
discloses a method for calculating the above-mentioned 
amount of feature. 

According to a recent endoscopic image processing 
method, Gabor features calculated using a well-known Gabor 
filter are processed by a spatial frequency analysis 
technique, that is improved for the application to 
endoscopic images, so that the fineness of the surface 
structure of a mucosa or the orientation of a pattern on the 
surface structure thereof is digitalized as the amount of 
feature . 

Various amounts of feature obtained from different 
findings are combined into a feature vector. More complex 
and correct diagnosis can be supported using the feature 
vectors. To increase the precision of the endoscopic 
diagnosis support system, a method for calculating the 
amount of feature with high precision to digitize a 
significant finding based on an endoscopic image is very 
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important . 

Japcuiese Unexamined Patent Application Piiblicatlon No. 
10-14864 discloses an exeunple of the above-mentioned 
endoscopic diagnosis support system and a method for 
calculating the amount of feature. 

Significant findings for diagnoses using endoscopic 
observations include the shape and size of a lesion, the 
color tone of a mucosa, a visible blood vessel image, and 
the surface structure (pattern image including pits) of a 
mucosa. The present invention relates to an image 
processing method for a visible blood vessel image and the 
surface structure of a mucosa among the above findings. 

Diseases, diagnosed using a visible blood vessel image 
as an important finding, include ulcerative colitis. The 
visible blood vessel image is important to determine the 
level of inflammation or the degree of remission. 

Due to the appearance of a magnifying endoscope in 
which high image quality is realized and high resolution of 
a solid-state image pick-up device (CCD, or the like) is 
also realized, which has the same diameter and operability 
as those of a normal endoscope and further has a zooming 
function, a very minute capillary on the surface of a mucosa 
and the pit pattern of a stomach or a colon can be clearly 
observed through a recent endoscope system with the 
magnifying endoscope. 
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Living tissue can be observed using an endoscope in a 
clinical examination at the same level as that of a case 
where a tissue specimen has conventionally been observed 
using a stereoscopic microscope. New diagnostics using 
those minute structure observation views are studied 
actively and are established in fields related to digestive 
tracts and bronchial tubes. 

The following examples are given: 

Diagnosis of an esophageal neoplastic lesion (adenoma, 
cancer) based on a pattern change in interpapillary 
capillary loop (IPCL) on an esophageal mucosa disclosed in 
Reference Docximent 1 (H. Inoue, MAGNIFICATION ENDOSCOPY IN 
THE SOPHAGUS AND STOMACH, Digestive Endoscopy, JAPAN 
ASTROENTEROLOGICAL ENDOSCOPY SCOCIETY, Vol. 13, Supplement, 
July 2001, pp. S40-S41) ; 

Diagnosis of an infection with Helicobacter pylori 
based on observations of gastric collecting venules 
disclosed in Reference Document 2 (K. Yagi, ENDOSCOPIC 
FEATURES AND MAGNIFIED VIEWS OF THE CORPUS IN THE 
HELICOBACTER PYLORI -NEGATIVE STOMACH, Digestive Endoscopy, 
JAPAN GASTROENTEROLOGICAL ENDOSCOPY SOCIETY, Vol. 13, 
Supplement, July 2001, pp. S34-S35) ; 

Diagnosis of a classification of mucosal atrophy and 
neoplastic lesion based on observations of gastric 
microvasculatures disclosed in Reference Dociiment 3 (K. Yao 
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et al . , MICROGASTROSCOPIC FINDINGS OF MUCOSAL MICROVASCULAR 
ARCHITECTURE AS VISUALIZED BY MAGNIFYING ENDOSCOPY, 
Digestive Endoscopy, JAPAN GASTROENTEROLOGICAL ENDOSCOPY 
SOCIETY, Vol. 13, Supplement, July 2001, pp. S27-S33) ; 

Diagnosis of a neoplastic lesion of a colon using pit 
pattern classification disclosed in Reference Document 4 
(Shin-ei Kudo, Depression Type Early Colonic Cancer, Nihon 
Medical Center, 1996, pp. 33-40) ; and 

Diagnosis of a bronchial infection and cancer based on 
observations of a microvasculature network in bronchi 
disclosed in Reference Document 5 (Kiyoshi Shibuya et al.. 
Endoscopic Observations of Bronchial Dysplasia using 
Magnifying Bronchial Video Scope, Vol. 22, No. 8, December 
2000, pp. 613-616) . 

On the other hand, those diagnoses based on endoscopic 
findings depend on subjective determinations of doctors. 
Disadvantageous ly, a variation in experience and knowledge 
between doctors may result in different diagnoses. It is 
desired that quantitative and objective diagnosis support 
information be provided using image processing. 

More specifically, the pattern of a blood vessel or 
pits is extracted from an image, the form, size, uniformity, 
or regularity of the pattern is digitized (called the amount 
of feature) by various image processing techniques, and the 
amoxint of feature is processed by a discrimination circuit 
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such as a linear discriminant function or a neural network. 
Thus, objective diagnosis support can be realized. 

Japanese Patent No. 2918162, to the Soune assignee, 
discloses an image processing and analysis method using a 
binarization process. 

However, the application of the image processing method 
with structure extraction by binarization to an endoscopic 
image has the following disadvantages. 

Endoscopic images are obtained with various observation 
distances and suigles. Further, in many cases, an 
observation subject has a curved form. Therefore, an 
endoscopic image has a large variation in brightness. In 
observing a pit pattern of a colon, generally, dye or a 
coloring agent, typified by indigo carmine or crystal violet, 
is sprayed in order to make pits clear. The density of dye 
may be varied or the dye may be sprayed unevenly (dye may be 
remained in interstitial portions other than pits) . 

Threshold processing is generally executed in the 
binarization process. For the above reason, however, it is 
difficult to use a fixed threshold value. Even when a 
threshold value is changed every image (it is studied in the 
character recognition field) , the extraction result is 
changed every local area of the image because a variation in 
brightness or the spray condition of dye. Disadvantageously, 
extraction leakage may occur. 
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The present invention is made in consideration of the 
above disadvamtages . It is an object of the present 
invention to provide an image processing method whereby a 
structural component such as a blood vessel image or a pit 
pattern serving as an extraction target can be favorably 
extracted in an (endoscopic) image. 

Another object of the present invention is to provide 
an image processing method whereby a structural component 
such as a blood vessel image or a pit pattern serving as an 
extraction target can be favorably extracted in an 
(endoscopic) image irrespective of image shooting conditions. 

Further, still another object of the present invention 
is to provide an image processing method for calculating the 
cimount of feature to realize high-precision diagnosis 
support information. 

Disclosure of Invention 

The present invention provides an image processing 
method including: an input step of inputting an image; and 
an extraction step of performing matching between a template, 
obtained by modeling a predetermined structural component in 
the image, and the image input in the input step to extract 
a structural component serving as an extraction target in 
the image. 
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Brief Description of the Drawings 

Pig. 1 is a block diagrcun showing the structure of a 
personal computer for executing an image processing method 
according to a first embodiment of the present invention. 

Fig. 2 is a flowchart explaining the details of a 
process by the image processing method according to the 
present embodiment. 

Fig. 3 is a diagram explaining a blood vessel image. 

Fig. 4 is a diagram explaining a blood vessel model 
according to the present embodiment. 

Fig. 5 is a diagram explaining an example of the 
formation of templates. 

Fig. 6 is a diagram explaining an example of the 
combination of one -dimensional directions in template 
matching. 

Fig. 7 is a diagram explaining a blood vessel image to 
describe the determination of a blood vessel position by 
template matching. 

Fig. 8 is a diagram explaining the determination of a 
blood vessel position by ten5>late matching. 

Fig. 9 is a flowchart explaining a series of processing 
steps according to a second embodiment of the present 
invention. 

Fig. 10 is a diagram explaining frequency 
characteristics of band pass filters. 
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Fig, 11 is a diagreun showing an example of an original 
image according to the present embodiment. 

Fig. 12 shows a first image example in process. 

Fig. 13 shows a second image example in process. 

Fig. 14 shows a third image example in process. 

Fig. 15 shows a fourth image example in process. 

Fig. 16 shows an example of a processed image. 

Fig. 17 is a flowchart explaining a series of 
processing steps according to a third embodiment of the 
present invention. 

Fig. 18 is a flowchart showing the details of binary 
image formation in step S24 in the flowchart of Fig. 17. 

Fig. 19 is a diagram explaining an example of the 
characteristics of bcuid pass filters. 

Fig, 2 0 includes diagrams explaining findings of blood 
vessels to be processed by an image processing method 
according to a fourth embodiment of the present invention. 

Fig. 21 is a flowchart explaining the details of a 
series of. processing steps according the fourth embodiment 
of the present invention. 

Fig. 22 is a diagram explaining the amount of feature. 

Fig. 23 is a flowchart explaining the details of a 
process according to a fifth embodiment of the present 
invention. 
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Best Mode for Carrying Out the Invention 

Embodiments of the present invention will now be 
described below with reference to the drawings. 
(First Embodiment) 

A first embodiment of the present invention will now be 
described with reference to Figs, 1 to 8. 

Figs, 1 to 8 relate to a first embodiment of the 
present invention; Fig. 1 a block diagram explaining the 
structure of a personal computer for executing an image 
processing method according to the present embodiment; Fig. 
2 a flowchart explaining the details of the image processing 
method according to the present embodiment; Fig. 3 a diagram 
explaining a blood vessel image; Fig. 4 a diagram explaining 
a blood vessel model according to the present embodiment; 
Fig. 5 a diagram explaining an example of the formation of 
templates; Fig. 6 a diagram explaining an excunple of the 
combination of one -dimensional directions in template . 
matching; and Figs. 7 and 8 diagrams explaining the 
determination of a blood vessel position by template 
matching. 

The first embodiment of the present invention relates 
to the image processing method that is effective in 
extracting an image of a branching blood vessel in, 
particular, an endoscopic image. The image processing 
method according to the present embodiment will now be 
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described below with reference to the drawings. 

For a visible blood vessel image in an endoscopic image, 
blood vessels of different thickness may be observed such 
that they constitute a tree. This tree includes a thick 
blood vessel called a drainage blood vessel and thin blood 
vessels branching of f therefrom. 

These blood vessels are different in thickness and 
depth in a mucosa. Accordingly, the images of the different 
blood vessels are also different in contrast from each other 
(generally, as a blood vessel is located deeper, the image 
thereof becomes more blurred) . A series of extraction 
processing steps, which will be described below, is applied 
to the above-mentioned blood vessel image. The image of a 
blood vessel serving as a structural component to be 
extracted is favorably extracted, thus providing objective 
diagnosis support. 

In the description of the present invention, it is 
assumed that an endoscopic image includes three (color 
component) images of R, G, and B each having a size (the 
nximber of lateral and longitudinal pixels) of ISX x ISY. 
Each of the R, G, and B images provides an 8 -bit gray scale 
having 0 to 255 levels. 

The image processing method according to the present 
embodiment is realized as a program rxinning on a personal 
computer. Fig. 1 is a block diagram of the arrangement of a 
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personal computer and peripheral devices used in the present 
embodiment . 

A personal computer 1 includes arithmetic means 2 
serving as a CPU, a main memory 3 which is used to store 
data and is used as a work area, and program storage means 4 
(such as a hard disk (abbreviated to HD in Fig. 1) in the 
present embodiment) to store a program. The personal 
computer 1 is connected to display means 5 such as a CRT, 
input means 6 such as a mouse and a keyboard, and external 
storage means 7 such as a magneto-optical disk (MO) . 

The personal computer 1 is also connected to an 
external network 9 such as a LAN (Local Area Network) via a 
network adapter 8. The external network 9 is connected to 
an endoscope system 11 for endoscopy and an image filing 
system 12 for filing image data of endoscopy. 

The personal computer 1 can obtain image data from the 
image filing system 12 or the endoscope system 11 through 
the external network 9, 

In the personal computer 1, the program stored in the 
program storage means 4 is read into the main memory 3 and 
is then executed by the arithmetic means 2 . 

Image data to be processed can be obtained as a file 
offline from the external storage means 7 or online via the 
external network 9. If the program storage means 4 has 
enough capacity, image data can also be stored in the 
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personal computer 1 in a manner similar to the program. 

According to the present embodiment, for example, the 
hard disk constituting the program storage means 4 stores 
templates of blood vessel models. As will be described 
later, the blood vessel models are obtained by modeling 
structural components to be extracted from input images. 

The CPU constituting the arithmetic means 2 performs 
arithmetic processing to calculate the amount of correlation 
between each pixel in an image and a template using the 
templates, and determines the position of a pixel with 
higher correlation in decreasing order of correlation, thus 
extracting a structural component. In other words, a 
structural component as an extraction subject is extracted 
by template matching. In this case, varying the values of 
parameters of each template provides a plurality of 
different templates so that subjects with different forms 
and sizes can be extracted. 

Fig. 2 is a flowchart explaining the details of the 
image processing method according to the present embodiment. 

In step SI, the arithmetic means 2 of the personal 
computer 1 receives a G image (component) serving as a 
captured original image from the image filing system 12. In 
general endoscopic images (without using dye or a coloring 
agent) , each structural component such as a blood vessel 
image or a pattern on the surface of a mucosa is mainly 
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produced by light absorption fluctuations of blood in the 
mucosa. 

More specifically, structural components mostly depend 
on the absorption band (wavelength) of hemoglobin in blood. 
It is known that many structural components are generally 
included in G images. Specifically, as the eunount of blood 
is larger, light absorption is higher, so that the values of 
pixels of a G image are lowered. As the amount of blood is 
smaller, light is not absorbed, but is reflected. Thus, the 
values of pixels are increased. 

In step S2, noise reduction is performed as 
preprocessing to random noise. According to the present 
embodiment, well-known median filtering (rearranging pixels 
including a target pixel in a mask in decreasing order of 
pixel value and replacing the value of the target pixel with 
a median value) is performed to each area of a size of 3x3 
pixels . 

Subsequently, in step S3, inverse y correction is 
performed to the image. y correction is non-linear 
processing to give a visually linear scale to an image when 
the image is displayed on a monitor. The inverse y 
correction converts the scale into the original linear one. 

In step S4, blood vessel extraction by matching using 
templates, naunely, by template matching is performed. The 
details of template matching in step S4 will now be 
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described below. 

Referring to Fig. 3, when fluctuations in pixel value 
are extracted and cut out in, for exanple, one dimensional 
direction (x direction) , from a blood vessel image in an 
endoscopic image, the fluctuations can be modeled 
(hereinbeiow, referred to as a blood vessel model) into a 
shape similar to a semiellipse (or a semicircle) 
(hereinbeiow, simply referred to as a semielliptic shape) 
shown in Fig . 4 . 

Therefore, a template having values W and D as 
parameters to define the semielliptic shape is formed. 
Pixels with high correlation are extracted to determine the 
position of a blood vessel. Thus, the blood vessel can be 
extracted. The width of a blood vessel Ccui be determined 
from the parameter W of the corresponding template. When x 
denotes a coordinate of a pixel viewed in the one- 
dimensional direction and y denotes a pixel value, a 
semielliptic function in Fig. 4 can be expressed by the 
following expression: 

Where, W and D denote parameters to define the width and 
depth of the semielliptic blood vessel model in Fig. 4, and 
k denotes a fixed real niimber. For example, k is set to 15 
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in an endoscopic image. In addition, actual endoscopic 
images have various blurs due to out-of -focus . Accordingly, 
smoothing (filtering by convolution operation in a digital 
image, the description of convolution operation being 
omitted because it is a well-known technique) using a 
Gaussian function is performed to the blood vessel model 
obtained from Expression (1) . 

Where, a denotes a standard deviation in the Gaussian 
function and is used as a parcuneter to determine the level 
of blur. As the value of a is larger, the effect of 
increasing the level of blur is enhanced. Further, 
according to the present embodiment, in order to eliminate 
an influence of a variation in brightness of an image in 
matching, which will be described later, scale conversion 
(gray level transform) in the y direction using logarithmic 
transformation is performed. 

Although a variation in the eunount of light upon image 
capturing affects as a multiplication term on a pixel value, 
the variation is separated as an addition term by 
logarithmic transformation, thus reducing the influence. 
Consec[uently, the precision of blood vessel extraction can 
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be increased. In other words, when scale conversion with 
logarithmic transformation is used, a blood vessel image can 
be favorably extracted almost independent of shooting 
conditions. 

Subsequently, the formed blood vessel model is 
subjected to discretization so that the model is used as a 
template for template matching in step S4 . According to the 
present embodiment, the blood vessel model having continuous 
function values is temporarily discretized such that ten 
subelements correspond to one element of a template (one 
element corresponding to one pixel of an image) . Further, 
the average of the values of the ten sxabelements is 
calculated, thus forming a discrete template (averaging 
produces an effect that fluctuations between elements of a 
template are smoothed) • 

In addition, since actual blood vessel images have 
various widths (sizes) and depths (magnitudes of variations 
of pixel values) , various combinations of W and D in value 
are formed to produce a plurality of templates. For a 
variation in the level of blur, templates can be added by 
changing the value of a. 

Fig. 5 shows examples of templates obtained by changing 
the values of W and a. In this instance, D is fixed to 50. 
Fig. 5 shows templates with various combinations in which W 
= 1 to 5 (step 1.0) and a = 0.0 to 4.0 (step 1.0). The size 
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of template is properly changed depending on the values of W 
and a. According to the present embodiment, a margin of one 
pixel is provided at a pixel position of y = 0.0 on each of 
both the ends of each template. 

Template matching using the above-mentioned templates 
will now be described. Matching is executed such that the 
cJDove -mentioned templates are superimposed on each pixel of 
an image f as in the case of convolution operation and 
normalized cross -correlation R expressed by the following 
expression is calculated (the description of normalized 
cross -correlation is omitted because it is well known in the 



Where, N denotes the size of a template, fx(i) denotes 



from the image so as to correspond to the template, fxM 
denotes the average of fx(i), t(i) indicates the value of 
the i-th element of the template, and tM denotes the average 
of t(i) . For the value of R, 0<R<1. As the value of R is 
closer to 1, the degree of matching is higher. 

In an actual endoscopic image, blood vessels run in 



art) : 




(3) 



the i-th pixel of N pixels which are one-dimensionally cut 
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various directions. For the directions of templates and 
cut-out images, for excunple, four directions of 0®, 45**, 90"^, 
and 135^ shown in Fig. 6 are used. Normalized cross- 
correlation in Expression (3) is obtained with respect to 
the four directions, thus obtaining RO, R45, R90, and R135. 

For example, referring to Fig. 7, a blood vessel image 
exists in the longitudinal direction of an image. When the 
width thereof corresponds to five pixels, the template of 
W=5 is applied to the image in the direction 0^. The 
normalized cross -correlation RO (W=5) is the highest at the 
blood vessel midsection at which the blood vessel matches 
the template. Fig. 8 shows the values of RO (W=5) with 
respect to one -dimensional shift in the template. Fig. 8 
shows a distribution in which x at which R=1.0 corresponds 
to a pixel positioning at the center in the width direction 
of the blood vessel and the value of x is reduced stepwise 
each time x is shifted from the center by one pixel. 

In the above-mentioned manner, for example, template 
matching is performed in M directions using K templates, so 
that KxM normalized cross-correlations R are obtained every 
pixel. When D is set to a fixed value, the magnitude 
relation between the normalized cross -correlations R 
obtained by respective templates is not changed. In the 
following description, therefore, for the sake of simplicity, 
D is set to a fixed value* 
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In step S5 (Fig, 2) , whether a blood vessel having a 
target pixel at the center exists is determined on the basis 
of the normalized cross -correlation R calculated in step S4. 
The determination processing is performed as follows. 

Obtained KxM values of Rij (l<i<K, l<j^M) are 
compared to each other, thus obtaining the maximixm value 
Rmax. Subsequently, the maximum value Rmax is s\ibjected to 
threshold processing. On the basis of a threshold value Th 
(for example, Th=0.9), if Rmsu^Th, the degree of matching 
between a blood vessel model, having parameters W and a 
corresponding to the maximum value Rmax, and an applied 
template in the corresponding direction 0 is high. Thus, it 
is determined that there is a blood vessel having the target 
pixel at the center in the width direction. 

When it is determined that the blood vessel exists, the 
process proceeds to step S6. If NO, the process proceeds to 
step S7. In step S6, the determination result in step S5 is 
stored into the external storage means 7. In this case, for 
example, the result of the following table a[k] is stored. 

First, a table a[ISXxiSY] having the same size of 
ISXxiSY as that of the image is prepared. Values of all 
cells are initialized to zero. 

From the results of steps S3 and S4 for the k-th pixels 
xk, if a template and direction in which Rmax>Th exist, a 
value t it^O) to specify the template and direction is set 
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to the corresponding table cells a [k] . According to the 
present embodiment, t indicates continuous values, for 
example, l<t<KxM with respect to KxM normalized cross- 
correlations Ri j . When it is determined that a blood vessel 
does not exist, the value in each table cell a[k] is set to 
0 as it is. 

In step S7, whether all pixels in the image are 
completely subjected to processing steps in steps S4 to S6 
is determined. If all the pixels are not processed, the 
process is returned to step S4 . The next pixel (from the k- 
th pixel xk to the (k+1) -th pixel x(k+l)) is again subjected 
to the processing steps in step S4 to S7. If all the pixels 
are completely processed, the process proceeds to step S8. 

In step S8, a blood-vessel extraction image is formed 
from the blood vessel extraction result produced in step S6. 
Specifically, the value of the table cell a[k] (l<k<ISXxiSY) 
corresponding to each pixel is referred. If a[k]9t0, a line 
segment with the width W in which the pixel xk is located at 
the center is assigned in accordance with the width W of the 
template corresponding to the value of a[k] and the used 
direction 9. 

The result of assignment of line segments to all cells 
a[k] indicates a blood-vessel extraction image. In program 
implementation, an area b[ISXxSIY] having a size of ISXxiSY 
is prepared for the formation of a blood-vessel extraction 
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image. All pixels are initialized to zero. Values 
corresponding to line segments are set to 1. 1 is set to 
cells of the corresponding pixels. 

By the above-mentioned series of processing steps, a 
blood vessel in an endoscopic image can be favorably 
extracted independent of the thickness (width) of the blood 
vessel and the level of blur. 

As post -processing, general expansion or contraction 
processing is applied to the blood-vessel extraction image 
formed in step S8, so that more smoothed image can be 
obtained . 

In the formation of the blood-vessel extraction image 
in step S8, it may be determined that a blood vessel exists 
in both the adjacent pixels. The pixel having the larger 
maximum value Rmax is selected and the other pixel is 
rejected, so that the correct result can be obtained. In 
this case, the area b[ISXxiSY] to store the values Rmax of 
respective pixels is prepared in addition to the table 
a[ISXxlSY] . Values are set in step S6. 

As mentioned above, according to the image processing 
method of the present embodiment, a blood-vessel extraction 
image can be favorably f canned from an endoscopic image. 

So long as an image has a multiple gray scale, for 
example, an X-ray angiographic image can also be processed 
in addition to endoscopic images. Further, so long as a 
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Structural component has a linear shape (irrespective of 
length) , any subject can be processed. Accordingly, a 
siibject to be extracted is not limited to a blood vessel. 

According to the present embodiment, a series of 
processing steps including template matching is applied to 
all the pixels of an image. To reduce processing time, the 
above series of processing step csui be performed every 
several pixels. 

There is a high probeJDility that a pixel having a low 
value corresponds to a blood vessel. Therefore, pixels 
having a lower value are previously obtained locally by 
retrieving. The obtained pixels alone can be processed. In 
this case, when a target pixel is positioned at the center 
of an area of, for example, 3x3 and the target pixel has the 
minimum value, the target pixel can be set to a subject to 
be processed. 

Therefore, the present embodiment has the following 
advantages . 

According to the present embodiment, a blood-vessel 
extraction image can be favorably formed from an endoscopic 
image . 

(Second Embodiment) 

A second embodiment of the present invention will now 
be described below with reference to Figs . 9 to 16 . Figs . 9 
to 16 relate to the present embodiment; Fig. 9 a flowchart 
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explaining a series of processing steps according to the 
present embodiment; Fig. 10 a diagram explaining frequency 
characteristics of band pass filters; and Figs. 11 to 16 
diagrams explaining an original image, images in process, 
and a processed image according to the present embodiment. 

A series of image processing and analysis according to 
the present embodiment is executed as a program running on 
the personal computer 1 in a manner similar to the first 
embodiment. Since the stroicture of the personal computer 1 
is the same as that in Fig. 1, the description thereof is 
omitted. 

The present embodiment particularly relates to an image 
processing method that is effective in extracting the 
pattern of pits (hereinbelow, referred to as a pit pattern) 
of a colon in an endoscopic image. 

Methods for extracting a pit pattern from an endoscopic 
image to analyze the shape thereof utilize binarization 
using threshold processing for a band pass filtered image. 
Disadvantageously, stable extraction is hardly performed by 
setting a fixed threshold value or setting a threshold value 
every image because an endoscopic image may have a variation 
in brightness and a difference or unevenness in dye density. 
According to the present embodiment, an image processing 
method, whereby pits to be analyzed can be stably extracted 
from an image having the above-mentioned disadvantages, will 
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now be described. 

When threshold processing Is uniformly performed to an 
Image processed by beuid pass filtering (hereinbelow, 
abbreviated to BPF) or the entire region of interest, 
extraction leakage or discontinuity occurs due to a 
variation in brightness and an unevenness in dye. In 
extraction, therefore, it is necessary to locally determine 
a threshold value for pits to be extracted as appropriate. 
According to the present embodiment, each region including 
pits to be extracted is locally extracted and the optimum 
threshold value is set every region. 

The present embodiment utilizes the following 
properties of band pass filters: A high-frequency band pass 
filter is capable of obtaining an extraction image along the 
shape of a pit pattern but it is susceptible to noise. A 
low-frec[uency band pass filter obtains an extraction image 
that is larger than the actual shape but it is resistant to 
noise. According to the present embodiment, a region 
including an extraction target is separated from regions 
including other components such as noise, and a threshold 
value is locally set every region including an extraction 
target, thus favorably obtaining the extraction image. 

The image processing method will now be described on 
the basis of the flowchart shown in Fig. 9. In the present 
embodiment, an R image is used as an original image (step 
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Sll) . Fig. 11 shows an example of an endoscopic R image as 
an original image. Dye (medical agent) called indigo 
carmine is sprayed in pit pattern observation. The dye is 
deposited in pits as hollows on the surface of a mucosa. 
Since the dye has tints of blue and green, the pattern of 
pits is most effectively reflected in an R image. 

In step S12, noise reduction and inverse y correction 
are performed in a manner similar to steps S2 and S3 in Fig. 
2 in the first embodiment. 

Subsequently, in step S13, band pass filters BPF(Lo) 
and BPF(Hi) having different frequency band characteristics 
are used to obtain respective processed images. For the 
band pass filters BPF(Lo) and EPF (Hi), Laplacian-Gaussian 
filters are used. Relatively high (Hi) and low (Lo) pass 
frequency bands are set. The Hi -pass frequency 
characteristic is along the frecjuency band distribution of a 
pit pattern component to be extracted. The Lo-pass 
frequency characteristic is lower than Hi by one octave. 

The Laplacian-Gaussian filter is well known and is 
described in detail in Reference Document "Vision, David 
Marr, Sangyo Tosho, pp. 58-66". The detailed description 
thereof is omitted. 

Fig. 10 shows the frequency characteristics Flo and FHi 
of the band pass filters BPF(Lo) and BPF(Hi) used in the 
present embodiment (although the frecjuency space of an image 
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is essentially two-dimensional, the space is one- 
dimensional ly shown for the sake of simplicity) . 

In step S14, binarization processing is performed to 
each of the processed images obtained through the band pass 
filters BPF(Lo) and BPF(Hi) on condition that the threshold 
value is set to 0.0. The obtained binary images are set to 
ResLo cuid ResHi. If a pixel is extracted as a pit pattern 
component, 1 is given to the pixel. If NO, 0 is given to 
the pixel. 

According to the present embodiment, the dyed pit 
pattern component has a small pixel value as in the case of 
a blood vessel according to the first embodiment. Pixels, 
in each of which the result by the band pass filter BPF(Lo) 
or EPF (Hi) indicates 0 or lower are extracted as a pit 
pattern component. In this case, all of the pixels having a 
negative variation are extracted. Thus, there is no 
extraction leakage due to a variation in brightness and an 
unevenness in dye. 

Figs. 12 and 13 show examples of the binary images 
ResLo and ResHi, respectively. The binary images are 
obtained by binarizing the endoscopic R image, including the 
pit pattern shown in Fig. 11, in step S14 of Fig. 9. 

In step S15, logical operation is performed every pixel 
in each of ResLo cuid ResHi, thus extracting np regions 
(coupled pixel groups) Pi (l<i<np) including a pit pattern 
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component to be extracted and nn regions Nj (l<j<nn) such as 
noise. 

In this case, it is assiimed that labeling is performed 
to the binary images to assign niombers for identifying a 
region to the respective regions Pi and Nj . Pixels 
extracted in both of ResHi and ResLo are included in Pi. 
Pixels, which are extracted in ResHi but are not extracted 
in ResLo, are included in Nj (the pixels are designated by 
Not ResLo in Fig. 9) . 

Figs. 14 and 15 show image examples of Pi and N j . 

Subsequently, in step S16, components such as noise, 
which cannot be completely removed in step S15, are 
eliminated. Specifically, the average ^iHi of each Pi is 
obtained from the absolute values in the process result 
ResHi of EPF (Hi) . The average )iHei of the eliminated noise 
regions Nj included in the pixels of a size of sxs 
surrounding the center of gravity (cxi, cyi) of Pi are 
obtained from the above absolute values. The average ^H± is 
compared to the average |iHei. A pit pattern component has a 
larger variation than that of a noise component. 
Accordingly, for example, when |iHi<^Hei, the corresponding 
region is eliminated. 

In step S17, a proper threshold value is set every 
remaining Pi and threshold processing, namely, re- 
binarization is performed. For example, the discriminant 
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analysis method by Ohtsu is used to set a threshold value. 

After the execution of step S17, a noise region may be 
remained. Further, elimination processing is performed in 
step S18, In this instance, the same elimination processing 
as that in step S16 is again performed. In this case, an 
elimination reference is set to |aHi<wnHei (w>1.0), thus 
increasing the effect of elimination. 

Fig. 16 shows an example of an extracted image obtained 
by the processing in step S18 (the region of interest is 
further set and outlines and halation are eliminated) . 

According to the edDOve -mentioned series of processing 
steps, irrespective of shooting conditions, namely, even if 
an endoscopic image has a variation in brightness and a 
difference or unevenness in dye density, a pit extraction 
image can be favorably extracted from the endoscopic image. 

According to the image processing method according to 
the present embodiment, structural components other than 
pits, for example, a blood vessel image can be extracted. 

Therefore, the present embodiment has the following 
advantages . 

According to the present embodiment, even if an 
endoscopic image has a variation in brightness and a 
difference or unevenness in dye density, a pit extraction 
image can be favorably extracted from the endoscopic image. 

(Modification of Second Embodiment) 
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When there are regions Pi which cannot be eliminated 
because ResHi has no difference, elimination processing 
based on a difference in tint can be performed subsequent to 
step S18. 

Specifically, the following properties are utilized: 
Since dye is deposited in each pit, tints of blue and green 
components are enhanced. For example, the ratio of R/G is 
obtained every pixel from the R and G images, subjected to 
noise reduction and inverse y correction, of Pi and Pk (k^ti) 
surroxinding Pi (the B image is further used and the ratio of 
R/ (R+G+B) can also be used) • The average |irgi of each 
region is compared to the average j^rgk of each region. When 
^rgi>nrgk (the color of the region is red relative to the 
surrounding extraction regions) , the corresponding region is 
eliminated. 

In eliminating noise regions, conventionally, threshold 
processing (eliminating a small region) has been performed 
to the number of region pixels extracted by binarization. 
Elimination is performed based on tint information of a 
local pit pattern component, thus obtaining an extraction 
image without wrong elimination for very small pits. 

According to the above-mentioned series of processing 
steps, even if an endoscopic image has a variation in 
brightness and a difference or iinevenness in dye density, a 
pit extraction image can be favorably obtained from the 
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endoscopic image . 

The image processing method according to the present 
embodiment can also be performed to structural components 
other than pits, for example, a blood vessel image. 
(Third Embodiment) 

A third embodiment of the present invention will now be 
described with reference to Pigs. 17 to 19. Figs. 17 to 19 
relate to the third embodiment. Figs. 17 and 18 show 
flowcharts explaining a series of processing steps according 
to the present embodiment. Fig. 19 is a diagrcun explaining 
an example of the characteristics of band pass filters. 

The present embodiment will describe a method for 
setting the optimiim process parameters with efficiency in 
binarization processing for image analysis. A series of 
image processing and analysis in the present embodiment is 
executed as a program running on the personal computer 1 as 
in the case of the first embodiment. The structure of the 
personal computer 1 is the same as that in Fig . 1 . The 
description thereof is omitted. 

In binarization processing for extracting a structural 
component in an endoscopic image, it is necessary to set 
parameters such as the frequency characteristic of a band 
pass filter and various threshold values. Such adjustment 
is mainly performed on the basis of the subjective matching 
between a structural component such as a pit pattern or a 
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blood vessel and a binarization result. For instance, a 
plurality of band pass filters are used, binarization 
results are compared to an original image with respect to 
the presence or absence, size, and thickness of extraction 
leakage in each binarization result, and respective set 
threshold values are changed. 

In the above adjustment with trial and error, however, 
enormous parameters and the combinations thereof are set 
depending on the details of an image processing method. 
Disadvantageously, the burden of the operator is large. 

The present embodiment relates to a method for 
preparing a reference image to designate an extraction 
target structural component on an endoscopic image and 
setting the optimum parameters on the basis of the 
comparison of the degrees of matching between the reference 
image and binary extraction results based on the combination 
of parameters. 

A method for setting parameters in the present 
embodiment will now be described below with reference to 
Figs. 17 cuid 18. According to the present embodiment, an 
extraction target is set to a pit pattern of a colon sprayed 
with indigo carmine of an endoscopic image as in the case of 
the second embodiment. Extraction is performed using band 
pass filtering (BPF) and threshold processing for the 
filtered images. 
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First, in step S21, an R image is input as an original 
image. In a manner similar to step S12 of Fig. 9 according 
to the second embodiment, noise reduction and inverse y 
correction are performed in step S22 . 

Subsequently, in step S23, a plurality of parameters 
are generated. In other words, as process parameters for 
forming binary images in step S24, N (1<N) band pass filters 
having different frequency characteristics and P threshold 
values for extraction are set. 

For the frequency characteristics of the N band pass 
filters Fi (l<i<N) , for example, if N=3, the peaks of 
respective pass bands are provided every octave as shown in 
Fig. 19. Referring to Fig. 19, for the sake of simplicity, 
Fi denotes filter characteristic. For the P threshold 
values Thj (l<j<P) , for example, Thl=0.0, Th2=-0.1, Th2=- 
0.2, and ThP= (-O.lxP) . in the following description of 

the present embodiment, P=5. 

Subsequently, in step S24, binary images Bij are formed 
using the parameters set in step S23. Fig. 18 shows the 
details of binary image formation processing in step S24. 

Referring to Fig. 18, in step S31, the R image 
subjected to the noise reduction and inverse y correction in 
step S22 is input. Subsequently, in step S32, the image is 
subjected to band pass filtering (abbreviated to BPF in Fig. 
18) using the N band pass filters Fi set in step S23. 
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In step S33, binarization processing using the P 
threshold values Thj set in step S23 is performed to the 
respective BPF-processed images obtained through the N band 
pass filters in step S32, thus forming the binary images Bi j . 

All of the NxP binary images Bij shown in step S34 are 
formed. After that, the processing in step S24 is completed. 
The process proceeds to step S25 in Fig. 17. 

In step S25, the degree of matching between extraction 
results is calculated with respect to the respective 
obtained NxP binary images Bij as follows. 

First, the operator (doctor who conducts a diagnosis) 
visually selects and prepares a reference image, in which an 
extraction target structural component is designated, to be 
compared to the original image. For the formation of the 
reference image, for example, general drawing (painting) 
software is used. The shape of a structural component (pit 
pattern of a colon in the present embodiment) is traced and 
painted in accordance with the state visually observed. The 
resultant image is used as a reference image. 

In this instance, as a paint color to specify a 
structural component, 0, 255, 255 (constituting cyan) are 
set to R, G, B values because this color is not generally 
found in living mucous membranes of endoscopic images. In 
the following description, S denotes a formed reference 
image . 
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Subsequently, the degree of matching between the 
extraction results of the reference image S and each of the 
binary images Bij is calculated. The calculation of the 
degree of matching of extraction utilizes an evaluation 
function based on the number of matching extracted pixels 
between the binary image Bij and the reference image S and 
the number of matching non-extracted pixels therebetween. 

According to the present embodiment, it is assumed that 
all pixels are extracted on the binary image Bij and pixels 
having values (0, 255, 255) are correctly extracted on the 
reference image S. The ratio of the extracted pixels to the 
whole pixels is obtained: 

aij= (Mij -Lij ) / (ISXxISY) (4) 

Where, Mij denotes the number of matching extracted pixels 
between the binary image Bij and the reference image S and 
Lij indicates the number of mismatching extracted pixels 
(the number of pixels which are extracted in one image but 
are not extracted in the other one) . 

Expression (4) expresses an evaluation function for the 
extraction result. As the value of aij is larger, the 
degree of matching is higher. If it is higher, it is 
determined that the parameters are proper. aij is 
calculated every Bij. After that, the process proceeds to 
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Step S26. For the degree of matching extracted pixels and 
the degree of matching non-extracted pixels, a function with 
weighting may be used depending on which degree of matching 
has a higher priority. According to the present embodiment, 
both the degrees of matching are equally evaluated. 

In step S26, the respective aij are compared to each 
other, thus specifying i and j to obtain the maximvim aij . 
The corresponding band pass filter Fi and the corresponding 
threshold value Thj are determined as process parameters. 

An endoscopic image other than the reference image is 
subjected to binarization processing using the process 
parameters set as mentioned above. Thus, a favorably 
extracted image can be obtained without setting complex 
parameters by trial and error. 

A variation in contrast due to a fluctuation in 
brightness based on shooting conditions and a variation in 
size of a structural component in an image due to a 
difference in observation distance or magnification may be 
generated between images. Thus, the proper threshold value 
and the frequency characteristic of a band pass filter may 
be changed. On the assumption of the above situation, 
typical scenes corresponding to various conditions are 
prepared as a plurality of reference images. Parameters 
corresponding to the various conditions are set cuid are 
selectively used. Thus, a favorably extracted image can be 
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similarly obtained. 

Therefore, the present embodiment has the following 
advantages . 

According to the present embodiment, a favorably- 
extracted image can be obtained without setting complex 
parameters by trial and error. 

(Fourth Embodiment) 

A fourth embodiment of the present invention will now 
be described with reference to Figs. 20 to 22. Figs. 20 to 
22 relate to the fourth embodiment. Fig. 20 shows diagrams 
explaining blood vessel findings to be processed by an image 
processing method according to the present embodiment. Fig. 

21 is a flowchart explaining the details of a series of 
processing steps according to the present embodiment. Fig. 

22 is a diagram explaining the amount of feature according 
to the present embodiment. 

The present embodiment will describe an image 
processing method for calculating the amount of feature that 
is effectively used in determining a lesion from an 
extraction image obtained by binarizing a structural 
component in an image. A series of image processing 
according to the present embodiment is executed as a program 
running on the personal computer 1 as in the case of the 
first embodiment. The structure of the personal computer 1 
is the same as that in Fig. 1. The description thereof is 
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omitted. 

In the description about the calculation of the amount 
of feature according to the present embodiment, it is 
assiimed that processing is performed to a binary image in 
which a structural component has already been extracted. 

For example, as a blood vessel finding in an endoscopic 
image, the irregular shape of a blood vessel may be 
important in diagnosis. Generally, as the degree of 
irregularity is higher, the level of malignancy or the grade 
of a lesion is higher. Such irregularity findings include a 
variation in width of a blood vessel. 

Fig. 2 0(A) shows an image of a normal blood vessel 
having an almost uniform width. Fig. 20(B) shows an image 
of a blood vessel of a lesion, the blood vessel with a 
variation in width. According to the present embodiment, 
the following series of processing is used to calculate such 
a variation in width as the amount of feature. 

The series of processing according to the present 
embodiment will now be described below with reference to Fig. 
21. First, in step S41, a blood vessel extraction binary 
image is input. According to the present embodiment, the 
blood vessel extraction binary image is formed by any of the 
binarization and extraction processings described in the 
first to third embodiments of the present invention. 

Subsequently, in step S42, labeling is performed to 
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assign numbers to respective extracted blood vessel images. 
In the present embodiment, it is assumed that k regions Pk 
(l<k<K) exist. 

Subsequently, in step S43, distance transform and 
skeleton processing is performed. The distance transform 
and skeleton processing is one of image analysis methods, 
which are generally called skeletons. This processing is 
used to extract information related to the width of a 
graphic and the feature of the shape thereof. The details 
of the distance transform eoid skeleton are disclosed in 
Reference Docximent "Introduction to Computer Image 
Processing edited by Hideyuki Tamura, Soken Shuppan, pp. 77- 
80". The description thereof is omitted. 

For example, referring to Fig. 22, distances Dm 
corresponding to the respective widths of a blood vessel can 
be obtained on the center line of a blood vessel image at M 
pixels Qm (l<m<M) on the center line by the distance 
transform and skeleton processing. 

M depends on the length of the blood vessel. If a 
binarized region is a circle, M=l. Actually, Mk pixels Qm 
and Mk distances Dm are calculated in the respective K 
labeled regions. Accordingly, they are set to Qkm and Dkm. 

Subsequently, in step S44, a width irregularity 
evaluation value V is calculated on the basis of the pixels 
Qm and the distances Dm on the center line obtained in step 
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S43. In the present embodiment, it is assumed that a 
standard deviation of the distances Dm of all Qm is set to 
an evaluation value. 



Where, ^iD denotes the average of Dkm. As the value V is 
larger, a fluctuation in width is also larger. This value 
can be processed by a discrimination circuit (such as a 
linear discriminant function or a neutral network) using 
supervised data calculated from various image groups of 
normal cases and lesions. 

As mentioned above, according to the series of image 
processing in the present embodiment, the amount of feature, 
which is useful in determining a lesion with irregularity, 
can be calculated from an extraction image obtained by 
binarizing a structural component in an image. 

In the above description, blood vessel images in 
endoscopic images are used as examples . According to the 
present embodiment, a s\ibject to be processed and analyzed, 
in which the amoxmt of feature related to the width is 
useful, is not limited to them. The present embodiment can 
also be applied to a pit pattern image of a colon or an X- 
ray angiographic image. 

Therefore, the present embodiment has the following 
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advantages. 

According to the present eit±>odiment , the amount of 
feature, which is useful in determining a lesion with 
irregularity, can be calculated from an extracted image 
obtained by binarizing a structural component in an image. 

(Fifth Embodiment) 

A fifth embodiment of the present invention will now be 
described with reference to Fig. 23. The present embodiment 
will describe an image processing method for calculating the 
amounts of feature which are useful in determining a lesion 
from a blood vessel image in an endoscopic image. More 
specifically, three R, G, and B images, constituting an 
endoscopic image, are sxibjected to image processing, 
respectively, thus obtaining a plurality of amounts of 
feature. The method is characterized by using the amounts 
of feature . 

Blood vessels running through a living mucosa are 
different from each other in depth. In other words, the 
blood vessels are broadly classified into thick blood 
vessels riinning in a deep region of the mucosa, blood 
vessels running in an intermediate region of the mucosa, and 
thin blood vessels running in a region close to the surface 
layer. For R, G, and B images constituting an endoscopic 
image, the R image has the longest wavelength of irradiated 
light cuid the B image has the shortest one. 



- 43 - 



It is known that light with a long wavelength arrives 
at the deepest region of a living mucosa cmd light with a 
short wavelength is absorbed in a region close to the 
surface later. Generally, image processing without dye is 
performed to a G image of a general endoscopic image. 
According to the present embodiment, for the above-mentioned 
reason, the amount of feature related to a blood vessel in a 
deeper region is calculated from an R image and the amount 
of feature related to a blood vessel in a superficial region 
is calculated from a B image. Thus, the present image 
processing method is very useful in classifying a lesion on 
the basis of blood vessel images. 

A series of image processing in the present embodiment 
is executed as a program running on the personal computer 1 
as in the case of the first embodiment. The structure of 
the personal computer 1 is the same as that in Fig. 1. The 
description thereof is omitted. 

In the description about the calculation of amounts of 
feature according to the present embodiment, it is assumed 
that processing is performed to binary images in each of 
which a structural component has already been extracted. 

The present embodiment will be described with reference 
to a flowchart of Fig. 23. First, in step S51, R, G, and B 
blood-vessel extraction binary images are input. R, G, and 
B images are subjected to any of the binarization and 
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extraction processings described in the first to third 
embodiments of the present invention, so that the blood- 
vessel extraction binary images in the present embodiment 
can be formed • Each image is subjected to a series of 
processing steps S52 to S54 . 

In steps S52 to S54, the ssune processing steps as those 
S42 to S44 in Fig. 21 described in the fourth embodiment are 
performed to the respective images to calculate width 
irregularity evaluation values Vr, Vg, and Vb of the R, G, 
and B images. 

In step S55, the obtained values Vr, Vg, and Vb are set 
to a feature vector related to the blood vessel images in 
the endoscopic image to be processed. Then, the process 
terminates . 

The feature vector (Vr, Vg, Vb) is supplied to a 
discrimination circuit (such as a linear discriminant 
function or a neural network) using supervised data 
calculated from various image groups of normal cases and 
lesions, thus obtaining information related to structural 
components concerned with blood vessels, which are different 
from each other in depth. Consequently, the classification 
result can be obtained with higher precision than the case 
using only Vg. 

In the present embodiment, for the amount of feature to 
be calculated, the width irregularity evaluation value V 
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explained in the fourth embodiment is described as an 
example. It is important that information of blood vessels, 
located in various depths of a mucosa, obtained by using 
light with different wavelengths is processed as the amounts 
of feature and the amounts of feature are used as a feature 
vector in discrimination processing. For example, therefore, 
if thinning a binary extraction image is used, other amounts 
of feature such as branch and cross points can be used. 

Therefore, the present embodiment has the following 
advantages . 

According to the present embodiment, information 
related to structural components concerned with blood 
vessels located in various depths of a living mucosa is 
obtained, so that the useful amounts of feature can be 
calculated. 

(Sixth Embodiment) 

A sixth embodiment of the present invention will now be 
described. The present embodiment will describe an image 
processing method whereby a blood-vessel extraction image 
can be favorably obtained even when a difference in tint of 
blood vessels is generated due to structural components 
included in three R, G, and B images constituting an 
endoscopic image . 

To extract blood vessel images, a G image is generally 
processed. The structural component of a blood vessel. 
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which is seen blue in an RGB (color) image, is reflected in 
an R image. Accordingly, the structural component of that 
blood vessel may not be favorsJaly obtained in the G image. 
The above case may be caused by the difference in depth 
between blood vessels in a mucosa as in the case of the 
fifth embodiment of the present invention. 

According to the present embodiment, extraction 
processing is performed to R and G images, respectively. 
The extraction results are combined with each other. Thus, 
a favorable blood-vessel extraction image can be obtained. 

In the present embodiment, BPF-processed images, 
obtained by band pass filtering R and G images, and a 
binarization and extraction process using threshold 
processing are fundamentally used. Since the series of 
processing steps to obtain the BPF-processed images is 
common in the other embodiments, the description thereof is 
omitted. 

In the following description, Br and Bg denote BPF- 
processed images of the R and G images, respectively. 
Pixels in Br and Bg are set to Br(x, y) and Bg(x, y) 
(l<x^ISX, l^y<ISY) . 

The following combined image Brg is formed from the 
basis of the BPF-processed images Br and Bg. 



Brg(x, y)=axBr(x, y)+pxBg(x, y) 



(6) 
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Where, a ouid p are defined as follows. 




a=0, p=l 



a=1.0, p=1.0 



if Br(x,y)> Thr, 



else (if Br (x,y) ^Thr) . 



(7) 



Where, Thr denotes a threshold value to determine 
whether a structural component reflecting a blood vessel 
image exists in Br. For example, Thr=-1.0 (since the 
structure of a blood vessel has a negative variation, the 
sign is negative) . 

Expression (7) expresses a coefficient for combination. 
When any structural component is not included in Br(x, y) , 
a=0, so that only the G image is used. When it is included 
in Br(x, y) , a=p=1.0, so that the structural components of 
both the R and G images are combined with each other. 

Using Expressions (6) cuid (7) , the combined BPF image 
Brg is obtained by completely extracting the structural 
components in the R and G images. Finally, Brg(x, y) is 
subjected to threshold processing, thus forming a binary 
image . 

As mentioned above, even when the difference in tint 
between blood vessels is caused by structural components 
included in respective images, a blood-vessel extraction 
result can be favoreJDly obtained by the image processing 
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method according to the present embodiment. 

The present invention also includes embodiments 
obtained by partially combining the cQDOve -mentioned 
embodiments . 

The present invention is not limited to the above- 
mentioned embodiments but many modifications and variations 
are possible within the scope of the invention without 
departing from the spirit of the invention. 

Industrial Applicability 

As mentioned above, according to the present invention, 
a structural component such as a blood vessel image or a pit 
pattern to be extracted can be favorably extracted, 

CROSS REFERENCE TO RELATED APPLICATION 

This application claims benefit of Japanese Application 
No. 2002-354290 filed on December 5, 2002, the contents of 
which are incorporated by reference in the description, 
claims, and drawings of this application. 



